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METHOD FOR THE WZ^TERM2^RKIN6 OF A VIDEO 
SEQUENCE WITH ADAPTIVE SELECTION OF THE 
WATERMARK INSERTION ZONE, CORRESPONDING 
DETECTION METHOD, DEVICES, DATA CARRIER AND 
5 COMPUTER PROGRAM 

FIELD OF THE INVENTION 

The field of the invention is that of the 
watermarking of video sequences. This watermarking 
consists of the insertion, into the video sequence, 

10 of a mark that is invisible to the user and designed 
to protect the contents of the sequence and enable 
its copies to be traced. 

More specifically, the invention relates to a 
method for the watermarking of a video sequence by 

15 the insertion of a binary mark on the motion vectors 
of the video. 

BACKGROUND OF THE INVENTION 

There are several known techniques to date for 
the watermarking of video sequences. 

20 One of these techniques consists of the 

application, to the video sequences, of watermarking 
methods similar to those used for the watermarking of 
still images. Indeed, it can be assumed that a video 
stream consists of a sequence of still images and 

25 that the "intra" images of the sequence can therefore 
be watermarked according to a scheme conventionally 
applied to the still images. 
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However, video sequences have many properties 
that can be advantageously exploited to develop a 
watermarking technique that is proper to them. 

These properties include the following aspects: 
5 ' - the rough size of a video sequence is far 

greater than that of a still image, so that the 

insertion space of a mark is far greater in a 

video sequence; 

as compared with still images, video sequences 
10 have a time dimension, which can be used for the 

insertion of the mark. 

Furthermore, video sequences have constraints 
different from those of still images. Thus: 

- the complexity of the watermarking scheme of a 
15 video sequence must be low enough to enable the 

insertion and detection of a mark to be done on 
the fly; 

the mark of a video sequence must be less 
visible than in the case of a still image, 
20 because the motion of the objects often 

increases its perceptibility to the user; 

- the video streams are often compressed in order 
to reduce the rough size of the sequences- The 
compression standard most commonly used is the 

25 MPEG2 standard, and the mark can be inserted 

directly into the format itself. However, 
insertion into the decompressed format should 
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not result in an increase in file size after 
compression; 

- video sequences can undergo more attacks than 
still images, and this often leads to the 
5 insertion of the mark or watermark redundantly 

so as to reduce the chances of success of these 
attacks, and make the watermark more robust. 

I 

When the watermark is inserted redundantly in 
the sequence, it may be estimated through a 

10 computation of an average value on all the 

images of the sequence. An additional 

constraint therefore is that it should be 
possible to detect the watermark or signature 
even after a loss of synchronization produced by 

15 the selection of a precise sub-sequence of the 

video sequence considered, or by the loss of 
images of the sequence. 

In addition to watermarking techniques from the 
field of still images, watermarking techniques proper 

20 to video have also been developed. These techniques 
make use of the advantageous properties of video 
sequences, in taking account of the constraints 
associated with them. . 

Among watermarking techniques specific to video, 

25 several approaches relying on the insertion of a mark 
into motion vectors have been proposed, especially by 
F.Jordan, M. Kutter, T. Ebrahimi in "Proposal of a 
watermarking technique for hiding/retrieving data in 
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compressed and decompressed video ISO/IEC document 
JTC1/SC29/WG11/ MPEG97/M2281, July 1997, and by J. 
Zhang, J. Li, L, Zhang in "Video watermarking 
technique in motion vector Proc. XIV Brazilian 

5 symposium on computer graphics and image processing, 
15-18.10.2001, pp 179-182. 

F. Jordan et al. have proposed a marking 
algorithm in which the mark is inserted into the 
motion vectors of a video. This technique relies on 
10 the implementation of the following steps: 

- a first step for the generation of the mark, 
which is a binary sequence having a length of 16 
bits or 32 bits; 

- a second step for the insertion of the mark. To 
15 do this, the motion vectors are extracted from 

the compressed video stream, obtained by an 
MPEG4 codec (or coder-decoder) . If the video 
stream is decompressed, there is a preliminary 
step of compression of this stream. The 

20 different marking steps are applied to one of 

the two components of the motion vectors. One 
block of pixels per image is selected randomly 
and its motion vector is computed. Two bits of 
the mark are inserted into each component of the 

25 motion vector. 

If V denotes the vertical component of a motion 
vector, and if h={0,l} is the value of the bit to 
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be concealed, then the following is the algorithm 
for the insertion of b into V: 
if ( (V^q-hT)mod[2] 

then V = V -h S 
5 else V = V 

with T = 2*dim 

dim = size of the search window for the motion 
estimation 

5 = (2,n+l)/q 

10 n^l if the motion vector is the null vector, 

n=0 else 

q = factor of modulation of the amplitude of the 
motion vector 

V is the marked motion vector. 

15 In other words ^ as many motion vectors are 

selected as there are bits in the mark, namely 16 
or 32. Then, for each of the 16 or 32 vectors 
selected, the corresponding bit of the mark is 
inserted into one of the components of the motion 

20 vector, for example the vertical component, in 

modifying its parity. 

a third step for the extraction of the mark, 
during which the motion vector marked V* is 
extracted from the compressed stream. The mark 
25 is then extracted as follows: 

b = (V'*q+T)modl2] 
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J. Zhang et al. propose an improvement of the 
above method proposed by F. Jordan et al., but their 
approach nevertheless remains the same. 

These two techniques therefore rely on the 
5 implementation of a watermarking algorithm, based on 
the insertion of a mark within the motion vectors of 
a video. 

However, a major drawback of these prior art 
techniques is that they do not have sufficient 

10 robustness with respect to possible attacks against 
video sequences. In particular, such methods are not 
sufficiently robust against non^malicious attacks 
such as compression or (spatial or temporal) changes 
in the format of the video signal. 

15 Indeed, the above techniques proposed by F. 

Jordan and al. or by J. Zhang et al. rely on the 
implementation of a watermarking algorithm, based on 
the insertion of a mark into the motion vectors of a 
video sequence, using the notion of parity for the 

20 ordinate value of these motion vectors. 

This approach is by definition not robust since 
the smallest attack can convert an even-parity 
ordinate value into an odd-parity ordinate value and 
vice versa. 

25 Furthermore, according to these techniques, it is 

generally always the same predetermined component of 
the selected motion vectors (for example the vertical 
component) that is modified. This has the effect of 



increasing the visibility of the mark in the video 
sequence. The visibility of the mark is of course 
disagreeable to the user who perceives a deformation 
of the image, 

5 In order to overcome these different drawbacks, 

the inventors of the present patent application have 
developed a new more robust technique for the 
watermarking of a video sequence described in the 
French patent application No. FR 02 13660 dated 31 

10 October 2002, filed by the present applicant and 
entitled "Precede de tatouage d'un signal video, 
systeme, support de donnees pour la mise en oeuvre de 
ce precede, precede d' extraction du tatouage d'un 
signal video, systeme pour la mise en oeuvre de ce 

15 precede" (Method for the watermarking of a video 
signal, system, data carrier for the implementation 
of this method, method for the extraction, from the 
watermark, of a. video signal, system for the 
implementation of this method) 

20 According to this technique, a motion vector that 

will carry the watermark is selected. The 
coordinates of the selected motion vector are 
identified in a space comprising a first plurality of 
zones associated with the binary value 1 and a second 

25 plurality of zones associated with the binary value 
0. Then, if necessary, the coordinates of the 
selected motion vector are modified so that it is 
located in the zone in which the binary value 
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corresponds to the value of the mark to be inserted 
into the selected motion vector. 

Thus, if it is desired to insert a mark with a 
value 0, and if the selected motion vector is located 
5 in a zone associated with the binary value 1, the 
coordinates of the motion vector are modified by 
obtaining a weighted central or axial symmetry 
relative to the edges of the zone. 

This technique proves to be therefore more robust 

10 than the methods proposed by F. Jordan and al. or by 
J. Zhang et al., because it defines the limited space 
in which the end of the motion vectors is located 
(zones with a binary value 1 or 0) . The watermarking 
made therefore no longer relies solely on a change in 

15 parity of the component of a motion vector but on the 
shift of the vector to be watermarked over a 
relatively large zone. 

As a consequence, even in the case of the 
filtering or light conversion of the image, leading 

20 to a modification of the coordinates of the 
watermarked motion vector, the end of this vector 
remains within the zone of the binary value chosen as 
a function of the mark. The mark is therefore robust 
against such attacks. However, although it is more 

25 robust, this technique is not optimal in terms of the 
invisibility of the mark. Indeed, the marked motion 
vectors cause deformation in the image which is 
sometimes troublesome for the user. 
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Furthermore, the robustness of this technique 

against more common attacks such as compression, 

cropping or mirror type attacks for example is not 

sufficient or could be improved. 
5 SUMMARY OF THE INVENTION 

It is a goal of the invention especially to 
overcome these drawbacks of the prior art. 

More specifically, it is a goal of the invention 
to provide a technique for the watermarking of a 

10 video sequence having increased robustness as 
compared with the prior art techniques. In 
particular, it is a goal of the invention to provide 
a technique of this kind that is particularly 
resistant to malicious attacks as well to more common 

15 attacks of the filtering or compression type. 

It is another goal of the invention to implement 
a technique of this kind that reduces the visibility 
of the mark or watermark as compared with prior art 
techniques. In particular, it is a goal of the 

20 invention to provide a video watermarking technique, 
according to which a mark is inserted into one or 
more motion vectors, and that makes it possible to 
minimize the deformations of the marked vectors as 
compared with the prior art techniques. 

25 It is yet another goal of the invention to 

provide a technique of this kind for the faster 
detection of a mark in a watermarked video sequence. 
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It is another goal of the invention to provide a 
technique of this kind that is simple to implement 
and costs little in terms of resources. 

These goals as well as others that will appear 
5 here below are achieved by means of a method for the 
watermarking of a sequence of video images 
implementing a step for the insertion of at least one 
watermarking bit on at least one motion vector 
obtained by motion estimation between two images of 

10 said sequence, so as to obtain at least one 
watermarked motion vector. The motion vector is 
identified by its coordinates in a reference space, 
partitioned into two types of complementary zones, 
each having a distinct binary value associated with 

15 it. If necessary, the insertion step' implements a 
modification of the coordinates of the motion vector 
so that it is located in a binary value zone 
corresponding to said watermarking bit to be 
inserted. 

20 According to the invention, during said 

modification, at least two potential watermarked 
motion vectors are determined and, from among said 
potential watermarked motion vectors, an optimal 
watermarked motion vector is selected according to at 

25 least one predetermined criterion, so that the 
modified coordinates of said motion vector are those 
of said optimal watermarked motion vector. 
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Thus, the invention relies on an entirely novel 
and inventive approach to the watermarking of video 
sequences that takes account of an intrinsic 
characteristic of video, that is motion. Indeed, the 
5 technique of the invention relies on the insertion of 
a mark or watermark into one or more motion vectors 
of a video sequence that implements a modification of 
the coordinates of the selected motion vector or 
vectors . 

10 Furthermore, the invention relies on an adaptive 

selection of the zone of insertion of the mark, and 
therefore constitutes an improvement of the technique 
described in the French patent application No. FR 02 
13660. Indeed, the coordinates of the watermarked 

15 motion vector are no longer obtained by simple 
central or axial symmetry relative to the boundaries 
of the two zones having binary values 0 and 1, but 
are those of an optimal watermarked motion vector. 
This optimal watermarked vector is chosen as a 

20 function of a predetermined criterion from among a 
plurality of motion vectors representing potential 
watermarked motion vectors belonging to different 
potential zones of insertion of the mark. 

The optimal watermarked motion vector belongs to 

25 an optimal zone of insertion of the mark which, when 
judiciously chosen, increases the force of the mark 
and its transparency relative to the prior art, and 
therefore improves its detection speed. 
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Thus, as compared with the technique of the 
application No. FR 02 13660, the operation for 
obtaining a watermarked motion vector is no longer 

limited to the application of a predetermined 
5 geometrical transformation to the selected motion 
vector- Rather, the operation considers a set of 
motion vectors representing potential watermarked 
motion vectors, among which a vector that can be used 
to obtain the best watermarking of a sequence (in 
10 terms of robustness, of transparency, and of speed of 
detection of watermarking for example) is selected. 

A level is therefore added in the decision-making 
process for the watermarking of the sequence, thus 
increasing both the robustness and the invisibility 
15 of the mark. 

Preferably, said predetermined criterion is a 
criterion of invisibility of said watermarking. 

Such a criterion can also be a criterion of 
resistance of the mark to certain types of attacks 
20 (involving compression or filtering for example) or 
again a criterion of speed of detection of the mark 
in a watermarked video sequence. 

Advantageously, said reference space is 
associated with a reference grid comprising blocks of 
25 predetermined dimensions, each of said blocks 
comprising a zone of each of said types. 

In a two-dimensional space, a reference grid of 
this kind may be a grid of the Cartesian space 
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associated with an orthogonal reference system^ 
constituted by a set of angular or square blocks. In 
a preferred embodiment of the invention, each block 
has smaller-sized sub-block, centered within the 
5 block, corresponding to one of the types of zone (for 
example the zone associated with a binary value 1) . 
The other zone (for example the zone associated the 
binary value 0) corresponds to the complementary 
value of the sub-block within the block, and is 
10 located between the sub-block and the boundaries of 
the block. 

Advantageously, since said motion vector is 
located in a block of said reference grid called a 
reference block, said potential watermarked motion 

15 vectors are searched for in a zone of said reference 
block having a binary value corresponding to said 
watermarking bit. 

Thus, the totality of the. zone of the block 
associated with the value 0 or 1 to be inserted into 

20 the selected motion vector is scanned; the scanning 
covers all the motion vectors belonging to this zone 
(or only some of them) , representing potential 
watermarked motion vectors, and among these, a choice 
is made as a function of the criterion chosen of the 

25 optimal watermarked motion vector. When the 

criterion is a criterion of invisibility of the mark, 
the optimal vector is the one that makes the mark 
least visible to the user. 
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Preferably, said potential watermarked motion 
vectors are also searched for in a binary value zone 
corresponding to said watermarking bit belonging to 
at least one block adjacent to said reference block. 
5 Indeed, the inventors of the present patent 

application advantageously make use of the fact that 
the optimal watermarked vector is not necessarily the 
vector whose coordinates are closest to those of the 
watermarked motion vector, 

10 When the chosen criterion is a criterion of 

invisibility of the mark, it is therefore possible 
that the optimal watermarked motion vector will not 
belong to the same reference grid block as the motion 
vector initially selected to carry the mark but that, 

15 on the contrary, it will be geographically at a 
fairly great distance. 

According to a preferred variant of the 
invention, said potential watermarked motion vectors 
are all the motion vectors located in said zone being 

20 searched. 

Thus, the operation considers all the motion 
vectors located in the zone or zones in which the 
optimal watermarked vector is searched for; it is 
thus possible to carry out a pixel-by-pixel scan of 

25 the totality of the search zone or zones belonging to 
one or more blocks of the reference grid to consider 
all the motion vectors which represent potential 
watermarked motion vectors. 
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Advantageously, said predetermined criterion is a 
criterion of optimization of a peak signal-to-noise 
ratio (PSNR) associated with the each of said 
potential watermarked motion vectors. 
5 The PSNR (or "Peak Signal-to-Noise ratio") is 

commonly used in image and video processing. It 
assesses the degree of fidelity of an image relative 
to a reference image: in this case, it enables for 
example the measurement of the degree of fidelity of 

10 a watermarked image of a video sequence relative to a 
non-watermarked original image. Among all the motion 
vectors representing potential watermarked motion 
vectors, the optimal watermarked vector is the one 
used to obtain the best PSNR. 

15 Preferably, since an image of said video sequence 

is associated with at least two hierarchical levels, 
said method implements a motion estimation on at 
least one pair of images of said sequence for at 
least one of said levels, called an estimated level 

20 so as to determine a set of motion vectors of said 
estimated level, and the motion vectors of a higher 
hierarchical level are obtained by computing an 
average of the associated motion vectors in said 
estimated level. 

25 The invention therefore also exploits the notion 

of a hierarchy of motion vectors, enabling a further 
increase in the robustness of the mark. For example, 
in a blockwise approach, an image of the video 
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sequence may be considered to be a set of four blocks 
of pixels of the image, themselves divided into four 
respective sub-blocks having a smaller number of 
pixels and so on and so forth until the unity sub- 
5 block having the size of one pixel. At each 
breakdown level, it is possible to associate a motion 
vector with the corresponding block or sub-block. 

Advantageously, a watermarking method such as 
this comprises a step for the selection, from among 

10 the motion vectors of the highest hierarchical level, 
of at least one motion vector at which said step for 
the insertion of a watermarking bit is implemented. 

For example, it is chosen to insert a 
watermarking bit into the motion vector of one in 

15 every two blocks at the highest hierarchical level. 

Preferably, said modification of the coordinates 
of said selected motion vector is also applied to the 
corresponding motion vectors of at least one of said 
lower levels, so as to perform a redundant insertion 

20 of said watermarking bit. 

Thus, when it is chosen to insert a watermarking 
bit into a motion vector of the highest hierarchical 
level, this insertion is also carried over into the 
corresponding motion vectors of the lower level. 

25 According to an advantageous characteristic of 

the invention, said motion estimation is of the 
"block matching" type. 
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A "block matching " technique of this kind is 
used to compute the motion vector associated with a 
block of pixels of the image. It evaluates the best 
motion vector used to reconstruct this current block 
5 from a block of the same size of the preceding image 
or a following image of the sequence, shifted by 
means of the motion vector. 

Preferably^ a watermarking method of this kind 
also comprises a step of motion compensation based on 
10 said watermarked motion vector or vectors so as to 
generate a watermarked video image sequence. 

This step enables the reconstruction of a 
watermarked video sequence, in the form of a 
succession of compensated images, obtained from the 
15 original images to which the optimal watermarked 
motion vectors are applied. 

According to a first advantageous variant of the 
invention, each of said motion vectors being 
associated with a region of said image, said motion 
20 compensation is implemented on all the regions of 
said image associated with watermarked or non- 
watermarked motion vectors. 

According to a second advantageous variant of the 
invention, said motion vectors being each associated 
25 with a region of said image, said motion compensation 
is implemented in the region or regions of said image 
with which said watermarked motion vector or vectors 
are associated, and an image of said watermarked 
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video image sequence comprises said regions on which 
said motion compensation is implemented and at least 
one original region. 

Thus, two methods can be envisaged within the 
5 framework of the invention. 

In the first method, the motion compensation is 
obtained by means of the set of motion vectors, 
whether watermarked or not watermarked, associated 
with the image to receive a watermarked version of 
10 the image considered. 

A second method is used to perform the motion 
compensation only for blocks of pixels of the image 
corresponding to the motion vectors modified by the 
watermarking operation and to leave the other image 
15 blocks considered unchanged in order to obtain an 
image and therefore a video sequence of higher 
quality. 

The invention also relates to a method for the 
extraction of a watermark from a sequence of video 
20 images watermarked according to the watermarking 
method described here above. 

According to the invention, such a method of 
extraction comprises a step for the extraction of at 
least one watermarking bit inserted into at least one 
25 watermarked motion vector, said extraction step 
comprising sub-steps of: 
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- motion estimation between two images of said 
sequence, so as to obtain said at least one 
watermarked motion vector; 

- analysis of the position of said watermarked 
5 motion vector in a reference space, partitioned 

into two types of complementary zones, each of 
which has a distinct binary value associated 
with it; 

- the assigning to said watermarking bit of the 
10 binary value of the zone in which said motion 

vector is located. 

Advantageously, said watermarking bit being 
inserted redundantly into said video image sequence, 
said method of extraction comprises a step for the 

15 computation of at least one correlation coefficient 
used to assign an index of trust to said extracted 
watermarking bit. 

The invention also relates to a device for the 
watermarking of a video image sequence, comprising 

20 means for the insertion of at least one watermarking 
bit into at least one motion vector obtained by 
motion estimation between two images of said 
sequence, so as to obtain at least one watermarked 
motion vector, said motion vector being identified by 

25 its coordinates in a reference space, partitioned 
into two types of complementary zones, each of which 
has a distinct binary value associated with it, said 
insertion means comprising means, activated if 
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necessary, for the modification of the coordinates of 
said motion vector so that it is located in a binary 
value zone corresponding to said watermarking bit to 
be inserted. 

5 'According to the invention, said modification 

means comprise means for determining at least two 
potential watermarked motion vectors and for the 
selection, from among said potential watermarked 
motion vectors, of an optimal watermarked motion 
10 vector, according to at least one predetermined 
criterion, so that the modified coordinates of said 
motion vector are those of said optimal watermarked 
motion vector. 

The invention also relates to a device for the 
15 extraction of a watermark from a sequence of 
watermarked video images according to the 
watermarking method described here above. According 
to the invention, such a device comprises means for 
the extraction of at least one watermarking bit 
20 inserted into at least one watermarked motion vector, 
said extraction means comprising means of: 

motion estimation between two images of said 
sequence, so as to obtain said at least one 
watermarked motion vector; 
25 - analysis of the position of said watermarked 

motion vector in a reference space, partitioned 
into two types of complementary zones, each of 
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which has a distinct binary value associated 
with it; 

- the assigning to said watermarking bit of the 
binary value of the zone in which said motion 
5 vector is located. 

The invention also relates to a digital data 
carrier that can be read by a terminal, comprising 
means for the storage of the signal representing a 
watermarked sequence of video images according to the 
10 watermarking method described here above. 

The invention also relates to a computer program 
comprising program code instructions for the 
execution of the steps of the watermarking method 
described here above when said program is executed on 
15 a computer. 

The invention also relates to a computer program 
product comprising program code instructions recorded 
in a carrier that can be used in a computer, 
comprising: 

20 - computer-readable programming means to perform a 

step for the insertion of at least one 
watermarking bit into at least one motion vector 
obtained by motion estimation between two images 
of said sequence, so as to obtain at least one 

25 watermarked motion vector, said motion vector 

been identified by its coordinates in a 
reference space, partitioned into two types of 
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complementary zones ^ each of which has a binary 
value associated with it; 

- computer-readable means to perform a step for 
the modification, if necessary, of the 
coordinates of said motion vector so that it is 
located in a binary value zone corresponding to 
said watermarking bit to be inserted; 

- computer readable means to perform a step for 
determining at least two potential watermarked 
motion vectors and a step for the selection, 
from among said potential watermarked motion 
vectors, of an optimal watermarked motion vector 
according to at least one predetermined 
criterion, so that the modified coordinates of 
said motion vector are those of said optimal 
watermarked motion vector. 

The invention finally relates to a signal 
representing a video image sequence watermarked 
according to the watermarking method described here 
above . 

BRIEF DESCRIPTION OF THE DRAWINGS 

Other features and advantages of the invention 
shall appear more clearly from the following 
description of a preferred embodiment, given by way 
of sample, illustrative and non-restrictive example, 
and from the appended drawings, of which: 

- Figure 1 is a block diagram of the different 
steps implemented by the method for the 
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watermarking of a video sequence of the 
invention; 

Figure 2 illustrates the hierarchical 
decomposition of an image into blocks; 
Figure 3 is an example of a reference grid 
partitioned into two types of zones, each 
associated with a distinct binary value, in 
which the motion vectors according to the 
invention are placed; 

Figure 4 describes the principle of searching 
for the set of potential watermarked motion 
vectors of a given search zone of the grid of 
figure 3; 

Figure 5 provides a more detailed illustration 
of the way in which all the pixels of the search 
zone of Figure 4 are scanned to determine the 
optimal watermarked motion vector; 
Figure 6 shows an example of a potential 
watermarked motion vector of the search zone of 
Figure 4; 

Figure 7 illustrates the use of the PSNR 

criterion used to determine an optimal 
watermarked vector in the context of a blockwise 
approach; 

Figure 8, commented upon in appendix 1, 
illustrates the watermarking technique described 
in the French patent application No. 02 13660. 
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DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

The general principle of the invention relies on 
the adaptive selection of the insertion zone of the 
watermarking bit. In other words, the general 
5 principle of the invention consists, in the context 
of a technique for the watermarking of a video 
sequence by the insertion of a mark into the motion 
vectors, in considering several motion vectors which 
represent potential watermarked motion vectors, 

10 belonging to different zones having the same binary 
value of a reference grid associated with the image, 
and in choosing, from among these vectors, an optimal 
watermarked motion vector enabling the optimizing of 
a predetermined criterion, for example a criterion of 

15 robustness or invisibility of the watermarking. 

The watermarking of video sequences can be seen 
to be a necessity to protect their contents. 

Indeed, the protection of video contents has 
hitherto been carried out through the existence of 

20 systems, known as proprietary systems, which however 
have shown their weaknesses. Such systems generally 
rely upon the existence of a secret which, once 
disclosed, can easily be misused and therefore no 
longer provides for any efficient protection of the 

25 video contents. This is the case for example of the 
CSS ("Content Scrambling System ") used for the 
protection of DVDs (or "Digital Versatile Disks") . 
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The utility of watermarking lies in the fact that 
the protection is invisible to the user and that the 
medium does not suffer conceptual deterioration (as 
in systems that encrypt the medium) : the data carrier 
5 on which the video sequence is recorded is therefore 
perfectly usable. 

Video watermarking in particular has applications 
in the following fields: 

- digital cinema for which it is sought, by 
10 watermarking, to prevent a malicious individual 

from recording a film projected on the screen; 

- the protection of DVDs, for which it is sought 
to prevent the spread of unauthorized copies, by 
watermarking technique that are more robust than 

15 the methods used hitherto to protect carriers of 

this kind; 

online cinema, which represent one of the future 
applications of high bit rate networks and is a 
system close to the present pay-per-view system. 
20 The watermarking prevents copying of films or 

enables distributed contents to be tracked. 
The watermarking algorithm proposed by the 
present invention relies on the hierarchical and 
adaptive insertion of a mark into the motion vectors 
25 of a video sequence. 

Its different steps are described henceforth in 
greater detail with reference to figure 1. 
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During a first step referenced 11, a mark with a 
given length N is generated in the form of a 
succession of N bits with a value 0 or 1. 

During a second step referenced 12, an estimation 
5 of motion between two images It and It' (t^f) for P 
pairs of the video sequence (II, .... , IM) with P<=M is 
performed. This motion estimation, implemented 
according to well-known technique (for example a 
"block matching" type of technique) , is used to 
10 compute a field of motion vectors between the images 
It and It'f for each of the P pairs of images 
considered. 

T vectors are then selected (13) from among all 
the motion vectors computed according to a 

15 predetermined criterion (for example determined by a 
Yarrow type of PRNG (pseudo-random number generator) 
process into which the bits of the generated mark 
(11) are inserted . 

For each vector selected, a search zone of a 

20 predetermined size (for example a zone constituted by 
all the zones of an adequate binary value of the 
block of the reference grid in which the selected 
vector is located, and of each of the blocks that are 
adjacent to it) is then scanned (14) to determine all 

25 the motion vectors that represent potential 
watermarked motion vectors corresponding to the 
selected vector. 
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Depending on a predetermined criterion (for 
example a PSNR type criterion of invisibility of the 
mark) , the optimal watermarked motion vector is 
selected (15) from among all the motion vectors that 
5 represent potential watermarked motion vectors. 

During a step referenced 16, the marked video 
sequence is finally generated by performing motion 
compensation relative to the optimal watermarked 
motion vectors. 
10 Each of the other steps referenced 11 to 16 here 

above shall now be described in greater detail. 

The mark generated during the step referenced 11 
may represent a piece of information (of the 
copyright or other type) and is transcribed into 
15 binary mode {-1,1} with a given length N. It is 
therefore constituted by N bits, each inserted into a 
motion vector of the image sequence- To increase the 
robustness of the mark, it may be inserted 
redundantly into the video sequence: in other words, 
20 several distinct motion vectors of this sequence may 
bear the same watermarking bit. 

To insert the mark redundantly, it is possible 
especially to carry out a motion estimation between 
two images It and It'(t ^t^) for P as pairs of the 
25 video sequence (II, IM) with P<M. The mark could 

then be inserted T/N times into the sequence. 

In a particular embodiment, the motion estimator 
used is of the "block matching" type and motion 
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estimation is performed between the images t and t+1 
of the sequence (respectively between the images t+1 
and t, depending on whether it is a front or rear 
motion estimation that is made) . 
5 It may be recalled that "block matching" is an 

operation wherein, for each block of pixels at the 
image considered, an assessment is made of the best 
motion vector enabling the reconstruction of this 
current block from a same-sized block of the previous 

10 or following image, shifted by means of the motion 
vector. To do* this, a search is made around the 
current block in the previous image or next image, in 
order to determine the motion vector that minimizes a 
classic cost function known as a DFD ("Displaced 

15 Frame Difference ") function representing the 
difference between the block in previous or following 
image displaced and the current block in the image 
considered, 

In a preferred embodiment of the invention, a 
20 hierarchical motion estimation of L levels is carried 
out for the marking (with L> 1) . A hierarchical 
approach of this kind (with the use of a plurality of 
imbricated levels) advantageously enables the 
spreading of the mark. 
25 To achieve this, the image is considered to be 

constituted by a set of blocks 21 sized 2Nx2N pixels, 
themselves divided into sub-blocks 22 with a size of 
NxN pixels for example, and so on and so forth. A 



-29- 



motion estimation is carried out on the NxN-sized 
blocks 22 in order to obtain the motion vectors 
associated with each of these blocks. An operation 
is then performed to take the average of the motion 
5 vectors obtained, in order to determine the motion 
vectors of the higher hierarchical level associated 
with each of the 2NxN-sized blocks 21. This 
operation for calculating the average may be 
continued for the 4Nx4N-sized blocks 23 and so on and 

10 so forth until the coarsest hierarchical level. 

The motion estimation is then carried out on a 
fairly fine hierarchical level, and from this fine 
estimation, the motion vectors associated with the 
coarsest hierarchical levels are determined. 

15 A particular embodiment of the invention uses two 

hierarchical levels and the initial blocks used 
during the motion estimation have a size of 4x4 
pixels. 

When all the motion vectors have been computed 
20 during the motion estimation step 12, the operation 
is located at the coarsest level of the hierarchy and 
a set of T motion vectors (and therefore a set of 
blocks with which they are associated) , on which the 
watermark will be inserted, is selected. 
25 The blocks are selected according to a predefined 

rule which may be deterministic or pseudo-random. In 
a particular embodiment, a Yarrow type pseudo-random 
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PRNG selection is used. It is the blocks of the 
coarsest level that are selected. 

After the selection 13 of the T motion vectors 
into which it is desired to insert the mark, a search 
5 14 is then made, with respect to each of these 
vectors, for all the motion vectors that represent 
corresponding potential watermarked motion vectors in 
a predetermined search zone. 

To do so, a reference grid illustrated in figure 
10 3 is used. 

In the embodiment shown in this figure, this grid 
is rectangular and is generated in the Cartesian 
domain. The x-axis corresponds to the rows of the 
images and the y-axis .corresponds to the columns. 
15 The grid consists of a plurality of HxK-pixel- 

sized rectangular blocks 30. A particular embodiment 
of the invention uses a square grid with H = K = 7 
pixels . 

Each block has two complementary zones Zl and Z2. 
20 The zone Zl is rectangular with a size of h x k 
pixels, and is centered in the block 30. The zone Z2 
is located at the periphery of Zl, between Z2 and the 
boundaries of the block. 

Preferably, the values of h and k are chosen such 
25 a way that the zones Zl and Z2 have the same area. 

The zone Zl is associated with the binary value 1 
and the zone Z2 with the binary value 0. 
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Each selected motion vector 31 is then placed on 
the grid, and the block 30, as well as the zone Zl or 
Z2 in which it is located is determined. 

If the vector 31 is located in the zone Zl, as 
5 shown in figure 3, and if it is desired to insert a 
watermarking bit with a value 1 into this vector 31, 
it is not necessary to shift the vector 31 by 
modifying its coordinates. 

If, however, it is desired to insert a 
10 watermarking bit with a value 0 into the vector 31, 
then the coordinates of the motion vector 31 have to 
be modified so that it is located in a zone Z2 of the 
grid associated with the binary value 0. 

Conversely, if the vector 31 is located in the 
15 zone Z2 and if it is desired to mark it with a value 
0, it is not necessary to modify it. If however it 
is desired to mark it with the value 1, then its 
coordinates must be modified to obtain a watermarked 
motion vector located in the zone Zl of the reference 
20 grid- 
Appendix 1, which is an integral part of the 
present patent application, provides a description 
with reference to figure 8 of the technique for 
watermarking a motion vector 31 described in the 
25 patent application No. FR 02 13660. 

According to this technique, the model of 
modification of the motion vectors 31 was a 
deterministic model. According to the present 
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invention, however, this watermarking technique is 
improved by broadening the search space of the 
watermarked motion vector. 

The present invention therefore consists of an 
5 improvement of the technique described in appendix 1 
in that, after the selection of the representative 
motion vectors designed to bear the watermark, the 
insertion zone (i.e. the zone in which the 
watermarked motion vector will be located after 

10 modification of its coordinates) will be adaptively 
selected in the neighborhood of the vector selected 
for the marking. 

Indeed, according to the technique that is the 
object of the patent application No. FR 02 13660, 

15 each selected motion vector is associated with a 
single reference point (corresponding to the end of 
the block of the reference grid closest to the end of 
the motion vector in which it is located) . The 
coordinates of the watermarked motion vector are 

20 determined directly according to predefined rules, as 
a function of the distance and position of the motion 
vector relative to this reference point: the 
watermarked vector is always located in a direct 
neighborhood of the associated reference point. 

25 According to the present invention however, a 

search is made for the optimal position of the 
watermarked motion vector, not only in the vicinity 
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of the reference point but also in a more or less 
large neighborhood of this reference point. 

This search enables the selection of a zone for 
the insertion of the mark (namely the zone in which 
5 the watermarked motion vector is located) that is 
optimal in terms of PSNR and robustness. 

In a particular embodiment shown in figure 4, the 
zone chosen as a search zone is a set 41 of 9 blocks 
centered on the block 42 (including this block 42) of 
10 the reference grid 43. In other words, apart from 
the block 42 in which the motion vector to be marked 
31 is located, a search is also made for the 
potential watermarked motion vectors in a search 
window three blocks wide, comprising all the blocks 
15 directly adjacent to the blocks 42. 

Naturally, within this search zone, the only 
motion vectors considered are those that represent 
potential watermarked motion vectors located in the 
zones Zl or Z2 corresponding to the watermarking bit 
20 to be inserted. 

Thus, as illustrated in figure 4, it is also 
possible to choose a 5x5-block search window 43 for 
example- The size of the search window can be 
parametrized. 

25 Instead of building the watermarked vector by 

simple symmetry, as described in appendix 1, during 
the step referenced 14, an exhaustive search is made 
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in the search zone 41 or 43 determined from the 

reference point associated with the motion vector 31, 
The step 15 for determining the optimal 

watermarked motion vector then consists in 
5 positioning the motion vector 31 on all the pixels of 

the search zone 41 or 4 3 which maximizes a criterion 

based on the computation of the PSNR. 

To do this, a scan is carried out on all the 8x8- 

pixel-sized blocks 30 of the search zone 41 or 43 (in 
10 modifying the coordinates of the motion vector 31 

accordingly), as illustrated in figure 5. 

The motion vector 31 is successively positioned 

on each of the pixels (81, 83 for example) of the 

search zone 41 (or 43) . Each pixel 81 (and 83 
15 respectively) scanned has an associated block 82 (and 

84 respectively) built in such a way that the pixel 

81 (and respectively 83) is located at the left-hand 

bottom of this block. The different blocks 82, 84 

are not separated and may overlap. 
20 It is these different blocks 82, 84 associated 

with the current pixel 81, 83 that are used to 

determine the PSNR, as explained here below with 

reference to figure 7. 

A variation Sd is then determined, the variation 
25 corresponding to the shift of the motion vector 31 

between each of the new 8x8-sized blocks 82, 84 of 

the region 52 of the reference grid and the original 

block 42, as illustrated in figure 6. 
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It will be noted that, in figure 6, the unmarked 
motion vector 31 is located in a zone Zl associated 
with the binary value 1 of the block 42. The motion 
vector 51, which represent a potential watermarked 
5 motion vector, is located in the zone Z2 associated 
with the binary value 0 of the region 52, which 
corresponds to the insertion of a watermarking bit 0. 

As illustrated in figure 7, each motion vector 
51, which represents a potential watermarked motion 
10 vector of an image n of the video sequence, when it 
is applied to an initial block 85, makes it possible 
to obtain a compensated block 61, 62 or 63 for 
example, of the next image n+1. 

The PSNR between the initial block 85 and each of 
15 the other compensated blocks 61, 62 and 63 is 
computed. 

It may be recalled that the PSNR is defined, in 

decibels, by the following formula: PSNR =^\0\og , , 

MSE 

where m is the maximum value that can be taken by a 
20 pixel (for example 255 if images in eight-bit gray 
levels are considered) , and where MSE designates a 
mean square error between two images I and I defined 
by: 

M5£=-^XZZ[%>^'0-%>''0f ^ images sized XxY of 

t X y 

25 a sequence of T frames. 

The optimal watermarked motion vector is the one 
that makes it possible, from the initial block 85, to 
obtain the block 61, 62 or 63 for which the highest 
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PSNR is obtained: indeed, this block is one used to 
obtain the most faithful representation of the 
original image. 

The variation 5d associated with this optimal 
5 watermarked motion vector is then applied to the 
motion vectors of the four 4x4-sized child blocks of 
the initial block 85- 

From the computation of PSNR for each of the 4x4- 
sized blocks, the average PSNR for all these blocks 
10 is deduced. 

When the optimal watermarked motion vector has 
been selected, the operation descends into finer 
levels of the hierarchy, in order to transfer the 
modifications of the coordinates of the motion vector 
15 to the vectors of the finest level, so as to obtain a 
spreading of the mark. 

The invention therefore implements a hierarchical 
approach to the watermarking method. 

When all the motion vectors selected during the 
20 step referenced 13 have been watermarked, they are 
used to generate (16) the marked video sequence in 
performing a compensation of motion. 

Two methods can then be envisaged: 
- according to a first method, a motion 
25 compensation is carried out on all the blocks of 

the image, whether these blocks are associated 
with watermarked or non-watermarked motion 
vectors; 
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- in the second method, a compensation is made on 
the blocks selected during the step referenced 
13, and this image is completed by the remaining 
original blocks, so as to obtain a higher 

5 quality image. 

According to the invention, the detection of a 
mark in a watermarked video sequence is done in two 
steps : 

- first of all, a method that is dual to the 
10 watermarking method described above is 

implemented in such a way as to extract the 
mark. This method consists in: 

o carrying out a motion estimation between 
pairs of images of the watermarked video 
15 sequence; 

o deducing a set of motion vectors on 4x4- 

sized blocks of the image; 
o taking the average of these motion vectors 
so as to determine the motion vectors 
20 associated with the 8x8-sized blocks; 

o placing these average motion vectors on the 

reference grid; 
o for each of these vectors, depending on the 
position in the zones Zl or Z2 of the grid, 
25 deducing the value 1 or 0 of the 

watermarking bit that it carries; 
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o refreshing the mark, in the form of a 
binary sequence of N bits, from all the 
watermarking bits extracted; 
a statistical correlation is then performed to 
determine a threshold of trust characterizing 
the presence or non-presence of the mark- 
Indeed, the mark may be inserted redundantly 
into the image (for example 2.5 times per 
image) , and a statistical correlation step of 
this kind will be used, for example, to 
determine whether the mark obtained is truly 
identical to the mark that had been inserted. 
The following two rules of correlation may be 
used: 



Correlation rule 1 



w 
¥ 
w 
w 



Extracted mark 
Average of W 
Original mark 
Average of W 



Correlation rule 2: 



8 



7 



: Number of images 
d(W,W) : Hamming distance 



In order to validate the detection phase, an 

accumulation is achieved of the correlation 
coefficients for the rule 1 in order to obtain an 
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index of trust for the entire video. Rule 2 
already represents an accumulation. 

Although the present invention has been 
described with reference to preferred embodiments, 
5 workers skilled in the art will recognize that 
changes may be made in form and. detail without 
departing from the spirit and scope of the invention. 
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APPENDIX 1 

Technique for the watermarking of a motion vector according to the patent 
application No. FR 02 13660 filed on behalf of the applicant for the present 

patent document 

5 The rule of insertion of the mark into a motion 

vector 31 OC is based on a reference grid illustrated 
in figure 8. 

This rectangular grid is generated in the 
Cartesian domain. The x-axis corresponds to the rows 

10 of the images and the y-axis to the columns. The 
grid is constituted by a set of HxK-pixel-sized 
elements E. With H, K=l, J (a particular mode of 

the invention uses a square grid with H = K = 7) 

For each motion vector computed during the motion 

15 estimation process, the motion vector OC 31 is placed 
on the grid and its reference point P 72 which 
corresponds to one of the vertices of the rectangular 
element E to which the vector OC 31 belongs is 
determined. To compute the reference point P 72 of a 

20 motion vector, the method chooses the point of the 
rectangular element E which minimizes a distance 
between the end C of the vector 31 and one of the 
four vertices of E. In a particular embodiment, the 
standard LI is used. 

25 The vector 31 is then modified according to the 

following rule: 

There is generated a rectangular 
element e included in E with a size hxk with 



h = H - 51 and k = K - 52 with 51 and 52 such 
that the area of e is equal to the area of E-e. 
The element e corresponds to the zone Zl, 
associated with the binary value 1 and the 
element E-e corresponds to the zone Z2, 
associated with the binary value 0; 

If it is desired to insert the bit 1 
and if the motion vector is in Zl, nothing is 
done. 

If it is desired to insert the bit 1 

and if the motion vector is in Z2, it is shifted 

to Zl. To do this, C is shifted into D in an 

axial direction: 
rx = Vx + S^-PC^-l cas4a 
ry = Vy + S,-PCy-\ casAb 

or according to a center symmetry B: 

ry = Vy + S,-PCy~\ 

where Vx and Vy respectively represent the x-axis 
and the y-axis of the vector OC 31, and V'x and 
V y represent the coordinates of the watermarked 
vector OD 71. 

In other words, when the point C is located at 
equal distance from the edges BP and FP of E, the 
vector OC 31 is shifted by weighted center 
symmetry with a center P, for example according 
to the following relationship: 
PD=-2/5PC. 



If, by contrast, the point C is closer to one of 
the boundaries of the zone Zl, the point D is 
obtained by achieving a weighted axial symmetry 
(for example with a coefficient 2/5) of the 
vector OC 31 relative to this boundary. 

If the bit 0 is to be inserted and 
if the motion vector is in Zl, it is shifted to 
Z2 in obtaining a weighted central symmetry with 
a center B. C is therefore shifted to D. 
In a particular embodiment, the weighting used 
leads to the following modification 

rx = Vx+s^-pc^-i 

Vy = Vy + S^-PCy-l 

It the bit 0 is to be inserted and 
if the motion vector is in Z2, nothing is done. 



